Predict and Surveil by Sarah Brayne

Author:Sarah Brayne
Language: eng
Format: epub
Publisher: Oxford University Press
Published: 2020-10-15T00:00:00+00:00

Reinforcing Inequality through Big Data

Biased Training Data and Hidden Feedback Loops

The surveillance scholar David Lyon describes surveillance as a process of “social sorting” in which people are classified and stratified into categories for differential treatment.13 With regard to crime, police surveillance classifies some people and places as high risk and others as low risk. The premise behind both person- and place-based predictive policing is that we can learn about the future from the past. That holds both for crime and for inequalities. When we hold up a mirror to the past, those inequalities are reflected into the future.14 And if our data are incomplete or biased, algorithms will not only mirror, but also amplify inequalities.

The “past,” for algorithms, is historical data—what is known as training data. Machines are trained to discover useful patterns of statistical relationships in existing data, then accumulate those sets of relationships into a model used to automate the process of predicting the future from new data.15 What a given model learns and what it is trained to look for in the future depends on the kinds of data and outcomes programmers provide. Consequently, an algorithm is only as good as its training data.

Let me be more concrete. Consider the person-based points system the LAPD uses to identify people who are high risk. Recall that individuals receive five points if they are on parole or probation, five points if they are documented as having a gang affiliation, five points for a violent criminal history, five points for a prior arrest with a handgun, and one point for every police contact. Officers are instructed to find reasons to stop the highest-point people in their patrol areas. But this process obviously leads to a feedback loop: if individuals have a high point value, they are under heightened surveillance and therefore have a greater likelihood of being stopped. Because they gain points for police contact, each time they are stopped, their point value rises. In that sense, the LAPD’s predictive models have created a behavioral loop: they not only predict events (e.g., crime or police contact), they also actually contribute to those events’ future occurrence. Put differently, the mechanisms for inclusion in criminal justice databases and risk assessment programs determine the surveillance patterns themselves. And because even arrests that do not result in charges or convictions count toward risk scores, the point system can create a ratchet effect in which surveillance is increased absent any evidence that it is warranted.

The point system is supposed to focus scarce police resources on the “hottest” offenders. It should also help avoid legally contestable bias in police practices. However, in the process of quantifying criminal risk, the system hides both intentional and unintentional bias in policing and creates a self-perpetuating cycle. As the legal scholar Frank Pasquale writes, “bias can embed itself in other self-reinforcing cycles based on ostensibly ‘objective’ data.”16 The data that feed the person-based predictive policing model include past stops, arrests, and classifications (e.g., gang affiliated or on parole or probation).

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.